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(54) Method and apparatus for multiplexing multi-view three-dimensional moving picture 



(57) An apparatus and method for multiplexing a 
multi-view three-dimensional moving picture according 
to a user's request is disclosed. The apparatus multi- 
plexes a multi-view three-dimensional moving pictures 
by receiving moving pictures and information inputted 
from multi-view cameras and generating multi-view en- 
coded streams corresponding to received moving pic- 
tures and information, receiving back channel informa- 



tion of the user, selecting necessary multi-view encoded 
streams according to the back channel information and 
multiplexing the selected multi-view encoded streams 
by a frame or a field having same time information. The 
present invention can effectively process the multi-view 
three-dimensional moving picture according to the dis- 
play mode and system environment requested from the 
user by simplifying a synchronization process. 
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Description 

Field of the Invention 

[0001] The present invention relates to an apparatus 
and method for multiplexing multi-view three-dimen- 
sional moving pictures according to a user's request and 
a computer readable recording medium storing instruc- 
tions for executing the method; and, more particularly, 
to an apparatus and method for multiplexing multi-view 
encoded streams outputted at the same time from an 
encoder to one stream in order to simplify a synchroni- 
zation-process in the multi-viewthree-dimensional mov- 
ing picture system. 

Description of Related Arts 

[0002] In general, a multi-view three-dimensional 
moving picture is a set of data containing information of 
images of moving object acquired at same time from 
multiple cameras located at different viewpoints. The 
multi-view three-dimensional moving picture of object 
can be displayed on a display device for providing a re- 
alistic view of moving object and a stereopsis of moving 
object to a user. The multi-viewthree-dimensional mov- 
ing picture is used in various application fields. However, 
mass size of image data, which needs to be transmitted, 
and complexity of a synchronization-process between 
images has been spotlighted as disadvantages of the 
multi-view three-dimensional moving pictures. 
[0003] The multi-view three-dimensional moving im- 
age can be displayed in various modes such as a two- 
dimensional display, a panorama display, a stereoscop- 
ic display and an auto-stereoscopic display and each 
display type requires its own data format. However, a 
conventional moving data processing system does not 
support to synchronize various data format of the multi- 
view three-dimensional moving picture according to the 
display modes. 

[0004] Specially, a conventional three-dimensional 
moving picture processing method for the stereoscopic 
display mode transforms images of multiple viewpoints 
to single image and displays the single image at a two- 
dimensional image process system. In the conventional 
method, a multiplexing process of multi-view encoded 
streams is not required since only one multi-view stream 
is outputted from the encoder. 

[0005] However, in case of processing the three-di- 
mensional moving picture without transformation to the 
two-dimensional moving picture, a new image process- 
ing method has been demanded for multiplexing multi- 
ple multi-view encoded streams outputted from the en- 
coder in order to minutely synchronize between multi- 
view three-dimensional moving pictures and transmit- 
ting the multiplexed streams. Moreover, it has been de- 
manded a multiplexing method for selecting and multi- 
plexing data required to display according to the display 
mode such as the two-dimensional display mode, the 



panorama display mode, the stereoscopic display mode 
and the auto-stereoscopic display mode and transmit- 
ting the multiplexed data. 

[0006] However, there is no image data processing 
5 apparatus and method for providing choices of display 
modes to user and for performing appropriate process- 
es of the multi-view three-dimensional moving pictures 
according to the user's selection of display mode. 

10 Summary of the Invention 

[0007] It is, therefore, an object of the present inven- 
tion to provide an apparatus and method for multiplexing 
a multi-view three-dimensional moving picture accord- 

15 ing to user's request and a computer readable recoding 
medium storing instructions for executing the method. 
[0008] In accordance with an aspect of the present in- 
vention, there is provided an apparatus for multiplexing 
a multi-view three-dimensional moving pictures, includ- 

20 ing: stream generating unit for receiving moving pictures 
and information inputted from a first predetermined 
number of cameras and generating the first predeter- 
mined number of multi-view encoded streams corre- 
sponding to the moving pictures and information; infor- 

25 mation receiving unit for receiving back channel infor- 
mation including viewpoint information and a display 
mode selected by a user; and multiplexing unit for se- 
lecting a second predetermined number of multi-view 
encoded streams among the first predetermined 

30 number of the multi-view encoded streams according to 
the back channel information and multiplexing the se- 
lected multi-view encoded streams each having the 
same time information, on a frame by frame basis or a 
field by field basis. 

35 [0009] In accordance with another aspect of the 
present invention, there is also provided a method for 
multiplexing a multi-view three-dimensional moving pic- 
tures, including the steps of: a) receiving moving pic- 
tures and information inputted from a first predeter- 

40 mined number of cameras and generating the first pre- 
determined number of multi-view encoded streams cor- 
responding to the moving pictures and information; b) 
receiving back channel information including viewpoint 
information and a display mode selected by a user; and 

45 c) selecting a second predetermined number of multi- 
view encoded streams among the first predetermined 
number of the multi-view encoded streams according to 
the back channel information and multiplexing the se- 
lected multi-view encoded streams each having the 

50 same time information, on a frame by frame basis or a 
field by field basis. 

[0010] In accordance with further another aspect of 
the present invention, there is also provided a computer 
readable recoding medium storing instructions for exe- 
55 cuting the method, including functions of: a) receiving 
moving pictures and information inputted from a first 
predetermined number of cameras and generating the 
first predetermined number of multi-view encoded 
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streams corresponding to the moving pictures and in- 
formation; b) receiving back channel information includ- 
ing viewpoint information and a display mode selected 
by a user; and c) selecting a second predetermined 
number of multi-view encoded streams among the first 
predetermined number of the multi-view encoded 
streams according to the back channel information and 
multiplexing the selected multi-view encoded streams 
each having the same time information, on a frame by 
frame basis or a field by field basis. 

Brief Description of the Drawings 

[0011] The above and other objects and features of 
the present invention will become apparent from the fol- 
lowing description of the preferred embodiments given 
in conjunction with the accompanying drawings, in 
which: 

Fig. 1 is a diagram illustrating a system for process- 
ing, transmitting and receiving a multi-view three- 
dimensional moving picture according to user's re- 
quest in accordance with a preferred embodiment 
of the present invention; 

Fig. 2 is a flowchart explaining a method for multi- 
plexing multi-view three-dimensional moving pic- 
tures according to the user's request in accordance 
with a preferred embodiment of the present inven- 
tion; 

Fig. 3 is a diagram illustrating the method for multi- 
plexing moving pictures in case of a two-dimension- 
al display mode in accordance with the present in- 
vention; 

Fig. 4 is a diagram illustrating the method for multi- 
plexing moving pictures in case of a panorama dis- 
play mode or an auto-stereoscopic display mode in 
accordance with the present invention; 
Fig. 5 is a diagram illustrating a method for multi- 
plexing moving pictures in case of the stereoscopic 
display mode in accordance with a preferred em- 
bodiment of the present invention; and 
Fig. 6 is a diagram showing a packetized transmit 
stream in accordance with a preferred embodiment 
of the present invention. 

Detailed Description of the Invention 

[0012] Other objects and aspects of the invention will 
become apparent from the following description of the 
embodiments with reference to the accompanying 
drawings, which is set forth hereinafter. 
[0013] Fig. 1 is a diagram illustrating a transceiverfor 
processing, transmitting and receiving a multi-view 
three-dimensional moving picture according to user's 
request in accordance with a preferred embodiment of 
the present invention. 

[0014] Referring to Fig. 1, the transceiver includes a 
transmitter 11 and a receiver 12. The transmitter 11 has 



an encoder 111 for encoding multi-view three-dimen- 
sional moving pictures acquired by N cameras, wherein 
the N is natural number bigger than 0, and outputting N 
multi-view streams, a multiplexer 112 for receiving N 

5 multi-view streams outputted from the encoder 111 , mul- 
tiplexing the N multi-view streams to one 3D_AU stream 
and outputting the 3D_AU stream and a packetizer 113 
for generating a header with additional information of N 
multi-view streams outputted from the encoder 111 and 

10 multiplexer 112, generating a payload with the 3D_AU 
stream outputted from the multiplexer 112 and packetiz- 
ing the header and payload to a transmission packet. 
[0015] The receiver 12 includes an unpacketizer 121 
for receiving the transmission packet from the transmit- 

15 ter 11 and unpacketizing the transmission packet to a 
header and payload, a demultiplexer 123 for demulti- 
plexing the 3D_AU stream included in the payload to 
multiple multi-view streams, an additional information 
analyzer 1 22 for analyzing the additional information in- 

20 eluded in the header and a decoder 125 for decoding 
the demultiplexed multi-view streams. 
[0016] As shown in Fig. 1, the encoder 111 receives 
N moving pictures of N viewpoints, encodes each of the 
received N moving pictures to a minimum near-by unit 

25 and outputs N multi-view encoded streams such as a 
stream_#1, a stream_#2, ... and a stream_#n, which 
represent an encoded stream of a first viewpoint, an en- 
coded stream of a second viewpoint, ... and an encoded 
stream of N th viewpoint. 

30 [0017] The N multi-view encoded streams outputted 
from the encoder 111 are inputted to the multiplexer 112. 
The multiplexer 112 selects necessary multi-view en- 
coded streams according to a display mode requested 
by the user and multiplexes the selected multi-view en- 

35 coded streams to a 3D_AU stream, as shown in Fig. 1 . 
The preferred embodiment of the present invention pro- 
vides fourdisplay modes such as a two-dimensional dis- 
play mode, a panorama display mode, a stereoscopic 
display mode and an auto-stereoscopic display mode. 

40 The user can select a desired display mode among the 
four display modes. According to the display mode se- 
lected by the user, necessary multi-view encoded 
streams are selected among N multi-view encoded 
streams for transmitting. 

45 [0018] Therefore, it is necessary to acquire back 
channel information of the userfor processing the three- 
dimensional moving pictures. The back channel infor- 
mation includes viewpoints and a display mode request- 
ed by the user. When the back channel information is 

50 received from the user, M multi-view encoded streams 
are selected according to the back channel information, 
wherein M is a natural number bigger than 0. The se- 
lected multi-view encoded streams are multiplexed. 
During multiplexing the selected multi-view encoded 

55 streams, the selected multi-view encoded streams can 
be multiplexed in a field or a frame by the multiplexer 
112. The method for multiplexing the selected multi-view 
encoded streams will be minutely explained with Fig. 2 
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in later. 

[0019] By multiplexing the selected multi-view encod- 
ed streams, a single 3D_AU stream is generated and 
the 3D_AU stream is inputted to the packetizer 113 for 
generating a transmission packet. The packetizer 113 
receives the 3D_AU stream, additional information from 
the encoder 111 and the back channel information of the 
user from the multiplexer 112. The additional informa- 
tion includes information of encoding method of the mul- 
ti-view encoded streams. The packetizer 113 generates 
a header by including the additional information and the 
back channel information of the user and generates a 
payload by including the 3D_AU stream. The packetizer 
113 finally packetizes the header and the payload to the 
transmission packet. 

[0020] The transmission packet is transmitted to the 
receiver 12. The transmission packet is unpacketized to 
a header and a payload in the unpacketizer 121. The 
header is passed to the additional information analyzer 
122 for analyzing the additional information and the 
back channel information of the user. The payload is 
passed to the demultiplexer 123 for analyzing and de- 
multiplexing the 3D_AU stream. The additional informa- 
tion and the back channel information of the user from 
the additional information analyzer 122 are necessary 
for demultiplexing the 3D_AU stream. 
[0021] M demultiplexed streams are passed and de- 
coded at the decoder 124. For decoding the demulti- 
plexed streams, the additional information from the ad- 
ditional information analyzer 122 is required. Finally, the 
decoded streams are passed to a display device and 
outputted according to the viewpoint and the display 
mode requested by user. 

[0022] Fig. 2 is a flowchart explaining a method for 
multiplexing multi-view three-dimensional moving pic- 
tures according to a display mode selected by a user in 
accordance with a preferred embodiment of the present 
invention. 

[0023] For implementing the preferred embodiment of 
the present invention, several assumptions of an envi- 
ronment are required. At first, input data of the preferred 
embodiment are data streams of three-dimensional 
moving pictures inputted from a plurality of cameras. For 
providing the data streams of three-dimensional moving 
pictures, at least two cameras are required. 
[0024] Cameras are arranged for capturing three-di- 
mensional moving pictures and it is distinguished with 
a camera arrangement for capturing two-dimensional 
moving pictures. For implementing the preferred em- 
bodiment of the present invention in order to support a 
two-dimensional display mode and a three-dimensional 
display mode, calibration information of each camera 
should be identical and cameras should be arranged 
with a difference of 65mm. 

[0025] The method for multiplexing multi-view three- 
dimensional moving pictures in accordance with the 
present invention requires viewpoint information and a 
display mode requested by a user since the present in- 



vention uses different method for multiplexing the multi- 
view three-dimensional moving pictures according to 
back channel information including the viewpoint infor- 
mation and the display mode requested from the user. 

5 [0026] The user can select the display mode among 
fourdisplay modes and the viewpointamong N numbers 
of viewpoints. For example, if the user wants to see two 
dimension moving pictures, then the user should select 
a two-dimensional display mode and a desired view- 

10 point among N numbers of viewpoints. After selecting 
the display mode and the viewpoints, the moving pic- 
tures of selected viewpoints will be displayed on a dis- 
play device after processing the moving pictures with 
appropriate methods according to the selected view- 

15 points and the display mode. That is, if the user wants 
the two-dimensional moving pictures, only necessary 
moving pictures are selected among moving pictures in- 
putted from the plurality of cameras and the selected 
moving pictures are processed for displaying on a dis- 

20 play device. Therefore, the user cannot see other sides 
of the moving pictures by turning the displayed images 
as like as three-dimensional computer graphic (CG). 
[0027] The present invention provides four display 
modes including a two-dimensional display mode, a 

25 panorama display mode, a stereoscopic display mode 
and an auto stereoscopic display mode. According to 
the display mode selected by the user, moving pictures 
of the viewpoints are differently processed. A method 
for processing the moving pictures according to each 

30 display mode will be explained in detail as follows 

[0028] At first, the two-dimensional display mode is a 
conventional moving picture display mode. If the user 
selects the two-dimensional display mode, moving pic- 
tures of the selected viewpoint, which are moving pic- 

35 tures inputted from only one camera, are multiplexed 
and transmitted to a receiver. 

[0029] Secondly, the panorama display mode re- 
quires the moving pictures of more than two viewpoints. 
That is, moving pictures of more than two viewpoints are 

40 provided to the user. For example, in case that the user 
selects the panorama display mode and desired view- 
points such as first, third and fifth viewpoints, encoded 
streams of the first, third and fifth viewpoints are multi- 
plexed to a 3D_AU stream and the 3D_AU stream is 

45 transmitted to a receiver. After processing the 3D_AU 
stream at the receiver side, the user is able to see the 
moving pictures of the first, third and fifth viewpoints. 
Therefore, it is a display mode that provides wider view 
of the moving picture. 

50 [0030] Third, the stereoscopic display mode is a mode 
for displaying three-dimensional moving pictures of two 
viewpoints acquired by two cameras. That is, in case 
that the user selects the stereoscopic display mode and 
desired viewpoints such as a first and a second view- 

55 points or a third and a fourth viewpoints, encoded 
streams of the selected viewpoints are multiplexed to a 
single 3D_AU stream and transmitted to a receiver. The 
transmitted 3d_AU stream is decoded and displayed on 
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a display device. 
[0031] In the above-mentioned stereoscopic display 
mode, an arrangement of cameras is a critical factor. 
For implementing the preferred embodiment for sup- 
porting the stereoscopic display mode, following as- 
sumptions are required. 

[0032] At first, there is a certain set of cameras that 
provides a moving picture as a stereo. When the user 
selects the viewpoints, it assumes that the user knows 
a pair of camera that supports the stereo. The user 
should select the viewpoints among the pair of camera 
that supports the stereo. For example, if the user selects 
a first viewpoint, then a second viewpoint should be se- 
lected and encoded streams of the first viewpoint and 
the second viewpoint are multiplexed to a single 3D_AU 
stream. The 3D_AU stream is transmitted to a receiver. 
[0033] The auto-stereoscopic display mode is a dis- 
play mode combining the panorama display mode and 
the stereoscopic display mode. 

[0034] The same assumptions of the stereoscopic 
display mode need to be implemented to the auto-ster- 
eoscopic display mode for selecting the viewpoints. 
That is, the user should select a certain set of viewpoints 
providing the moving pictures as the stereo. And the 
moving picture of each viewpoint is provided as the pan- 
orama display mode. For example, if the user selects a 
first viewpoint, a second viewpoint and a third viewpoint, 
then the first viewpoint and the second viewpoint should 
be provide the moving pictures as the stereo and the 
second viewpoint and the third viewpoint should provide 
the moving pictures as the stereo. And the moving pic- 
tures of each viewpoint are provided as the panorama 
display mode. Therefore, it is a display mode providing 
a three-dimensional view and a widerview of the moving 
pictures. 

[0035] In the auto-stereoscopic display mode, the 
method for multiplexing the multi-view three-dimension- 
al moving pictures of the panorama display mode is 
used since the moving pictures of three viewpoints 
should be multiplexed. 

[0036] Referring to Fig. 2 and the above-mentioned 
assumptions, the method for multiplexing the multi-view 
three-dimensional moving pictures according to a dis- 
play mode selected by user in accordance with the 
present invention is explained in detail as follows. 
[0037] The multi-view moving pictures from N multi- 
view cameras are inputted at a step 201 . 
[0038] The received multi-view moving pictures from 
the N cameras are encoded to each minimum near-by 
unit and N multi-view encoded streams are generated 
at a step 202, wherein the N multi-view encoded 
streams include a first viewpoint's multi-view encoded 
stream, a second viewpoint's multi-view encoded 
stream, ... , N th viewpoint's multi-view encoded stream. 
[0039] At a step of 203, back channel information of 
a user is analyzed for detecting a display mode and 
viewpoints selected by the user. 

[0040] According to a result of the step 203, multi-view 
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encoded streams corresponding to the viewpoints de- 
tected from the back channel information are selected 
among N multi-view encoded streams and the selected 
multi-view encoded streams are multiplexed according 

5 to the display mode selected by the user at steps 204, 
205 and 206. For example, if the user requests the two- 
dimensional display mode at the step 203, one encoded 
stream corresponding to a viewpoint requested by the 
user is multiplexed at the step 204. If the user requests 

10 the panorama display mode or the auto-stereoscopic 
display mode at the step 203, three encoded streams 
corresponding to viewpoints selected by the user are 
multiplexed at the step 205 and if the user requests the 
stereoscopic display mode at the step 203, two encoded 

15 streams corresponding to viewpoints selected by the us- 
er are multiplexed at the step 206. 
[0041] After multiplexing the multi-view encoded 
streams corresponding to the viewpoints selected by the 
user, the multiplexed stream is packetized with addition- 

20 al information of an encoding method and the user's 
back channel information to a transmission packet and 
the transmission packet is transmitted at a step 207. 
[0042] Figs. 3, 4 and 5 are a diagram explaining a 
method for multiplexing three-dimensional moving pic- 

25 tures according to the display mode requested from the 
user. 

[0043] Fig. 3 is a diagram illustrating the method for 
multiplexing three-dimensional moving pictures in a 
case of a two-dimensional display mode. 

30 [0044] Referring to Fig. 3, if the user requests the two- 
dimensional display mode, one encoded stream is se- 
lected according to a viewpoint requested from the user 
among received N numbers of encoded streams and the 
selected stream is multiplexed to a 3D_AU stream. 

35 [0045] Each encoded stream includes multiple num- 
bers of frames and each frame is multiplexed to the 
3D_AU stream. 

[0046] Fig. 4 is a diagram illustrating a method for mul- 
tiplexing three-dimensional moving pictures in a case of 
40 a panorama display mode or an auto-stereoscopic dis- 
play mode. 

[0047] Referring to Fig. 4, if the user requests the pan- 
orama display mode or the auto-stereoscopic display 
mode, multi-view encoded streams of a first viewpoint, 

45 a second viewpoint and a third viewpoint, which are re- 
quested from the user, are selected among the received 
N multi-view encoded streams and the selected multi- 
view encoded streams are multiplexed to a single 
3D_AU stream. 

50 [0048] That is, multi-view encoded streams of the 
three viewpoints selected by the user are orderly united 
and composed to the single 3D_AU stream and output- 
ted. In other words, the selected encoded streams hav- 
ing L numbers of frames are multiplexed to the 3D_AU 

55 stream by orderly multiplexing frames of selected three 
encoded streams to one frame. 

[0049] For transmitting the 3D_AU stream at the 
transmitter, additional information of only one encoded 
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stream is used. Therefore, unnecessary packet header 
can be eliminated, and efficiency of transmission in- 
creases. 

[0050] The same method for multiplexing the multi- 
view three-dimensional moving picture is required in 
both of the panorama display mode and the auto-stere- 
oscopic display mode; however, a method for demulti- 
plexing the received 3D_AU stream is not same in the 
panorama display mode and the auto-stereoscopic dis- 
play mode. 

[0051] Fig. 5 is a diagram illustrating a method for mul- 
tiplexing multi-view three-dimensional moving pictures 
in a case of a stereoscopic display mode in accordance 
with the present invention. 

[0052] Referring to Fig. 5, if the user requests the ster- 
eoscopic display mode and viewpoints of a first and sec- 
ond, encoded streams of the first viewpoint and the sec- 
ond viewpoint among the received N multi-view encod- 
ed streams are multiplexed to a 3D_AU stream. 
[0053] That is, multi-view encoded streams of two 
viewpoints are orderly united and composed to the sin- 
gle 3D_AU stream and outputted. In other words, the 
selected encoded streams having L numbers of frames 
are multiplexed to the 3D_AU stream by orderly multi- 
plexing frames of selected two encoded streams to one 
frame. 

[0054] In a receiver, a received 3D_AU stream is de- 
multiplexed to multi-view encoded streams and the mul- 
ti-view encoded streams should be decoded for display- 
ing moving pictures on a display device. For decoding 
the multi-view encoded stream at the receiver, informa- 
tion of encoding method is required. Therefore, the 
packetizer packetizes the necessary information for de- 
coding into a header by receiving additional information 
from the encoder and back channel information of user 
from the multiplexer. 

[0055] As mentioned above, by multiplexing multi- 
view encoded streams having identical time information 
to one basis stream, a synchronizing process of multi- 
view encoded streams can be simplified. Moreover, 
eliminating overlapped header information increases ef- 
ficiency of transmission. Time information in an encoded 
stream of single viewpoint is acquired and used for 
processing all other encoded streams. Therefore, over- 
lapped header information can be eliminated. 
[0056] Fig. 6 is a diagram showing a transmission 
packet in accordance with a preferred embodiment of 
the present invention. 

[0057] Referring to Fig. 6, a packetizer 1 1 3 packetizes 
additional information for an encoding method, a user's 
back channel information and a 3D_AU stream to a 
transmission packet and the transmission packet is 
transmitted to a receiver. 

[0058] The transmission packet includes a header 
and a payload. The header contains the additional in- 
formation for an encoding method and the back channel 
information of the user. The payload contains the 
3D_AU stream. 



[0059] The transmission packet is transmitted to the 
receiver and unpacketized to the header and the pay- 
load by an unpacketizer at the receiver. The header is 
passed to an additional information analyzer for analyz- 

5 ing the additional information and the back channel in- 
formation of the user. The 3D_AU stream included in the 
payload is demultiplexed at a demultiplexer. For demul- 
tiplexing, the additional information and the back chan- 
nel information from the additional information analyzer 

10 is required. 

[0060] The demultiplexed M encoded streams are 
passed to the decoder and decoded by using the addi- 
tional information included in the header. The decoded 
streams are passed to a display device and displayed 

15 according to the display mode and the viewpoints re- 
quested from the user. 

[0061] The above-mentioned method according to 
the present invention can be implemented as instruc- 
tions and stored to a computer readable recoding me- 
20 dium such as a CD-ROM, RAM, floppy disk, hard disk 
and optical magnetic disk. 

[0062] The present invention can effectively synchro- 
nize multi-view three-dimensional moving pictures ac- 
cording to four types of display mode. 
25 [0063] The present invention can process the multi- 
view three-dimensional moving pictures according to 
the display mode and system environment requested by 
the user. 

[0064] The present invention can simplify a synchro- 

30 nization-process by multiplexing multi-view encoded 
streams having identical time information to one basis 
stream and minimize overlapped header information by 
using time information acquired from an encoded 
stream of one viewpoint to synchronize encoded 

35 streams of all remained viewpoints. 

[0065] The present invention also increases an effi- 
ciency of transmission by streaming only data adaptable 
to the display mode selected by the user and an envi- 
ronment of a system while maintaining the comparability 

40 of a conventional system. 

[0066] While the present invention has been de- 
scribed with respect to certain preferred embodiments, 
it will be apparent to those skilled in the art that various 
changes and modifications may be made without de- 

45 parting from the scope of the invention as defined in the 
following claims. 

Claims 

50 

1. An apparatus for multiplexing multi-view three-di- 
mensional moving pictures, comprising: 

stream generating means for receiving moving 
55 pictures and information inputted from a first 

predetermined number of cameras and gener- 
ating the first predetermined number of multi- 
view encoded streams corresponding to the 



55 



6 



11 
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moving pictures and information; 
information receiving means for receiving back 
channel information including viewpoint infor- 
mation and a display mode selected by a user; 
and 

multiplexing means for selecting a second pre- 
determined number of multi-view encoded 
streams among the first predetermined number 
of the multi-view encoded streams according to 
the back channel information and multiplexing 
the selected multi-view encoded streams each 
having the same time information, on a frame 
by frame basis or a field by field basis. 

2. The apparatus as recited in claim 1, further com- 
prising packetizing means for generating a packet 
having a header and a payload and transmitting the 
packet, wherein the header includes the back chan- 
nel information and additional information, which 
contains information of encoding method of the mul- 
ti-view encoded streams generated in the generat- 
ing stream means, and the payload includes the 
multiplexed multi-view encoded stream. 

3. The apparatus as recited in claim 1 or 2, wherein 
the display mode includes a two-dimensional dis- 
play mode, a panorama display mode, a stereo- 
scopic display mode and an auto-stereoscopic dis- 
play mode. 

4. A method for multiplexing multi-view three-dimen- 
sional moving pictures, comprising the steps of: 

a) receiving moving pictures and information in- 
putted from a first predetermined number of 
cameras and generating the first predeter- 
mined number of multi-view encoded streams 
corresponding to the moving pictures and infor- 
mation; 

b) receiving back channel information including 
viewpoint information and a display mode se- 
lected by a user; and 

c) selecting a second predetermined number of 
multi-view encoded streams among the first 
predetermined numberof the multi-view encod- 
ed streams according to the back channel in- 
formation and multiplexing the selected multi- 
view encoded streams each having the same 
time information, on a frame by frame basis or 
a field by field basis. 

5. The method as recited in claim 4, further comprising 
a step d) for generating a packet having a header 
and a payload and transmitting the packet, wherein 
the header includes the back channel information 
and additional information, which contains informa- 
tion of encoding method of the multi-view encoded 
streams generated in the step a), and the payload 



includes the multiplexed multi-view encoded 
stream. 

6. The method as recited in claim 4 or 5, wherein the 
5 display mode includes a two-dimensional display 

mode, a panorama display mode, a stereoscopic 
display mode and an auto-stereoscopic display 
mode. 

10 7. The method as recited in claim 6, wherein if the two- 
dimensional display mode is selected, a multi-view 
encoded stream of one viewpoint selected by a user 
is multiplexed and transmitted. 

15 8. The method as recited in claim 6, wherein if the pan- 
orama display mode is selected, multi-view encod- 
ed streams of two or more viewpoints selected by 
the user are multiplexed and transmitted. 

20 9. The method as recited in claim 6, wherein if the ster- 
eoscopic display mode is selected, multi-view en- 
coded streams of two viewpoints selected by the us- 
er are multiplexed and transmitted. 

25 10. The method as recited in claim 6, wherein if the au- 
to-stereoscopic display mode is selected, multi- 
view encoded streams of two or more viewpoints 
selected by the user are multiplexed and transmit- 
ted. 

30 

1 1 . The method as recited in claim 6, wherein the head- 
er is generated by using only one among lots of ad- 
ditional information of multi-view encoded streams 
selected for multiplexing. 

35 

12. A computer readable recording medium storing in- 
structions for executing a method for multiplexing 
multi-view three-dimensional moving pictures, 
comprising functions of: 

40 

a) receiving moving pictures and information in- 
putted from a first predetermined number of 
cameras and generating the first predeter- 
mined number of multi-view encoded streams 

45 corresponding to the moving pictures and infor- 

mation; 

b) receiving back channel information including 
viewpoint information and a display mode se- 
lected by a user; and 

50 c) selecting a second predetermined number of 

multi-view encoded streams among the first 
predetermined numberof the multi-view encod- 
ed streams according to the back channel in- 
formation and multiplexing the selected multi- 

55 view encoded streams each having the same 

time information, on a frame by frame basis or 
a field by field basis. 
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